Method, system, and program for adding operations identifying data packets to structures based on priority levels of the data packets

ABSTRACT

Disclosed is a method, system, and program for adding an operation (e.g., an operation that provides information about data for transfer or a storage operation) to a structure (e.g., a queue). If a priority level associated with a data packet identified by the operation has a first designation comprising a high priority, placing the operation into a first structure (e.g., a queue) with a least number of operations. If the priority level associated with the data packet identified by the operation has a second designation comprising a low priority, placing the operation into a second structure (e.g., a queue) with a most number of operations.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a method, system, and program foradding operations identifying data packets to structures.

2. Description of the Related Art

In computer systems, components are coupled to each other via one ormore buses. A variety of components can be coupled to a bus, therebyproviding intercommunication between all of the various components. Anexample of a bus that is used for data transfer between a memory andanother device is the peripheral component interconnect (PCI) bus.

In order to relieve a processor of the burden of controlling themovement of blocks of data inside of a computer, direct memory access(DMA) transfers are commonly used. With DMA transfers, data can betransferred from one memory location to another memory location, or froma memory location to an input/output (I/O) device (and vice versa),without having to go through the processor. Additional bus efficientlyis achieved by allowing some of the devices connected to the PCI bus tobe DMA masters.

When transferring data using DMA techniques, high performanceInput/Output I/O controllers, such as gigabit Ethernet media accesscontrol (MAC) network controllers may be used. In particular, a hostcomputer includes an Input/Output (I/O) controller for controlling thetransfer of data packets to or from, for example, other computers orperipheral devices across a network, such as an Ethernet local areanetwork (LAN). The term “Ethernet” is a reference to a standard fortransmission of data packets maintained by the Institute of Electricaland Electronics Engineers (IEEE) and one version of the Ethernetstandard is IEEE std. 802.3, published Mar. 8, 2002.

To read a data buffer of a memory using DMA transfers, such as when thedata has to be retrieved from memory in response to a transmit commandfrom an operating system so that the data can be transmitted by the I/Ocontroller, a device driver for the I/O controller prepares the databuffer. A transmit command may be any indication that notifies thedevice driver of a data packet to be transferred, for example, over anetwork. The device driver writes one or more descriptors (i.e., thatinclude the data buffer's physical memory address and length, etc.) to acommand register of the I/O controller to inform the I/O controller thatone or more descriptors are ready to be processed by the I/O controller.The I/O controller then DMA transfers the one or more descriptors frommemory to another buffer and obtains the data buffer's physical memoryaddress, length, etc. After the I/O controller has processed the one ormore descriptors, the I/O controller can DMA transfer the contents/datain the data buffer.

A priority may be assigned to the data packets. For instance, for anEthernet LAN, data packets are assigned a priority ranging from level 0to 7, with 7 reflecting the highest priority level.

Some I/O controllers maintain one queue for storing high priority datapackets that are waiting to be transferred and another queue for storinglow priority data packets that are waiting to be transferred. Then, datapackets are selected from the two queues and transferred with a roundrobin technique (i.e., a data packet from the high priority queue isselected for transfer, then a data packet from the low priority queue isselected for transfer, a data packet from the high priority queue isselected for transfer, etc.) by the I/O controller. Moreover, it ispossible that low priority data packets may be transferred before queuedhigh priority data packets. For example, if the majority of data packetsare high priority, such as streaming audio or video data, the highpriority queue may have several high priority data packets waiting to betransferred. If a low priority data packet is then stored in the lowpriority queue, which has few or no other pending data packets, theround robin selection of data packets for transfer would select the lowpriority data packet for transmission before selecting a pending highpriority data packet.

This leads to a disruption of the transfer of high priority data packets(e.g., streaming audio or video data or protocol control packets).Therefore, there is a need for an improved technique for processing datapackets in queues.

BRIEF DESCRIPTION OF THE DRAWINGS

Referring now to the drawings in which like reference numbers representcorresponding parts throughout:

FIG. 1 illustrates a computing environment in which aspects of theinvention may be implemented.

FIG. 2 illustrates a format of a data packet in accordance with certainembodiments of the invention.

FIG. 3 illustrates logic implemented in a device driver in accordancewith certain embodiments of the invention.

FIG. 4 illustrates an example set of transfer operations for high andlow priority data packets being placed into queues in accordance withcertain embodiments of the invention.

FIG. 5 illustrates an example set of transfer operations for mostly highpriority data packets being placed into queues in accordance withcertain embodiments of the invention.

FIG. 6 illustrates an example set of transfer operations for mostly lowpriority data packets being placed into queues in accordance withcertain embodiments of the invention.

DETAILED DESCRIPTION

In the following description, reference is made to the accompanyingdrawings which form a part hereof and which illustrate severalembodiments of the present invention. It is understood that otherembodiments may be utilized and structural and operational changes maybe made without departing from the scope of the present invention.

FIG. 1 illustrates a computing environment in which aspects of theinvention may be implemented. A computer 102 includes a centralprocessing unit (CPU) 104, a volatile memory 106, non-volatile storage108 (e.g., magnetic disk drives, optical disk drives, a tape drive,etc.), an operating system 110, and a network adapter 112. The computer102 may comprise any computing device known in the art, such as amainframe, server, personal computer, workstation, laptop, handheldcomputer, telephony device, network appliance, virtualization device,storage controller, etc.

Any CPU 104 and operating system 110 known in the art may be used. Thenetwork adapter 112 includes a network protocol for implementing thephysical communication layer to send and receive network packets to andfrom remote devices over a network 116. The network adapter 112 includesan I/O controller 122. In certain embodiments, the I/O controller 122may comprise an Ethernet Media Access Controller (MAC) or networkinterface card (NIC), and it is understood that other types of networkcontrollers, I/O controllers such as small computer system interface(SCSI controllers), or cards may be used.

The network 116 may comprise a Local Area Network (LAN), the Internet, aWide Area Network (WAN), Storage Area Network (SAN), etc. In certainembodiments, the network adapter 112 may implement the Ethernetprotocol, token ring protocol, Fibre Channel protocol, Infiniband,Serial Advanced Technology Attachment (SATA), parallel SCSI, serialattached SCSI cable, etc., or any other network communication protocolknown in the art.

The storage 108 may comprise an internal storage device or an attachedor network accessible storage. Programs in the storage 108 are loadedinto the memory 106 and executed by the CPU 104. An input device 130 isused to provide user input to the CPU 104, and may include a keyboard,mouse, pen-stylus, microphone, touch sensitive display screen, or anyother activation or input mechanism known in the art. An output device132 is capable of rendering information transferred from the CPU 104, orother component, such as a display monitor, printer, storage, etc.

A device driver 118 includes network adapter 112 specific operations tocommunicate with the network adapter 112 and interface between theoperating system 110 and the network adapter 112. In particular, thedevice driver 118 controls operation of the I/O controller 122 andperforms other operations related to the reading of data packets frommemory 106. The device driver 118 may be software that is executed byCPU 104 in memory 106.

In addition to the device driver 118, the computer 102 may include otherdrivers, such as a transport protocol driver 128. The transport protocoldriver 128 executes in memory 106 and processes the content of messagesincluded in the packets received at the network adapter 112 that arewrapped in a transport layer, such as TCP and/or IP, Internet SmallComputer System Interface (iSCSI), Fibre Channel SCSI, parallel SCSItransport, or any other transport layer protocol known in the art.

In certain embodiments, the device driver 118 issues operations to theI/O controller 122. Although an operation may be any type ofinformation, command, etc., for examples described herein, the term“transfer operation” will be used to refer to an operation that providesinformation about data for transfer (e.g., across an Ethernet LAN).Other operations (e.g., a storage operation that is used to store datainto a structure) fall within the scope of the invention. An I/Ocontroller 122 maintains a first structure 124 (e.g., a queue) and asecond structure 126 (e.g., a queue) for storing the transferoperations. In certain embodiments, the device driver 118 issuestransfer operations to the I/O controller 122 and places the transferoperations in the structures 124, 126. The transfer operations identifydata packets stored in one or more data buffers 134. The I/O controller122 processes the transfer operations in structures 124, 126 to transferdata packets from data buffers 134 to a transfer structure 136 (e.g., aFirst In First Out (FIFO) queue) for transfer over, for example, network116.

Several of the devices of FIG. 1 maybe directly or indirectly coupled toa bus (not shown). For instance, the device driver 118 and the I/Ocontroller 122 maybe coupled to the bus.

Although structures/buffers 124, 126, 132, and 134 are illustrated asresiding in memory 106, it is to be understood that some or all of thesestructures/buffers may be located in a storage unit separate from thememory 106 in certain embodiments.

FIG. 2 illustrates a format of a network packet 250 in accordance withcertain embodiments of the invention. The network packet 250 isimplemented in a format understood by the network protocol, such as anEthernet packet that would include additional Ethernet components, suchas a header and error checking code (not shown). A transport packet 252is included in the network packet 250. The transport packet may 252comprise a transport layer capable of being processed by the I/Ocontroller 22, such as the TCP and/or IP protocol, Internet SmallComputer System Interface (iSCSI) protocol, Fibre Channel SCSI, parallelSCSI transport, etc. The transport packet 252 includes a priority level254 as well as other transport layer fields, such as payload data, aheader, and an error checking code. The payload data includes theunderlying content being transferred, e.g., commands, status and/ordata. The operating system may include a device layer, such as a SCSIdriver (not shown), to process the content of the payload data andaccess any status, commands and/or data therein.

The invention places transfer operations identifying data packets (e.g.,descriptors identifying the physical memory address and length of thedata buffers in which the data packets reside) into one of multiple(e.g., two) structures 124, 126 according to the priority level of thedata packet to be placed into one of the structures 124, 126 and thenumber of pending data packets already stored in each of the structures.High priority data packets are placed on the structure with the fewestnumber of pending data packets, and low priority data packets are placedon the structure with the highest number of pending data packets.

FIG. 3 illustrates logic implemented in a device driver 118 inaccordance with certain embodiments of the invention. Control begins atblock 300 with receipt of an operation identifying a data packet. Inblock 310, a priority level for the data packet is obtained. In certainembodiments, a priority level is included with the data packet. If apriority level is not already associated with the data packet, thepriority level of the data packet may be calculated in block 310 orbefore block 310 based on one or more factors, such as whether thepayload data includes audio or video data (e.g., audio or video data maybe high priority so that the audio or video data packets are senttogether to avoid disruption of an audio or video stream). In certainembodiments, the priority level may be associated with an alphabeticcharacter, symbol, numeric value, or other value.

In block 320, it is determined whether the priority level has a firstdesignation (e.g., a high priority level) or a second designation (e.g.,a low priority level), and if the data packet has a first designation,processing continues to block 330, otherwise, processing continues toblock 340. The first and second designations may be determined, forexample, by a system administrator. For example, if there are eightpriority levels, the top four priority levels (e.g., 4, 5, 6, and 7) maybe designated as high priorities, while the remaining four prioritylevels (e.g., 0, 1, 2, and 3) may be treated as low priorities. Incertain embodiments, block 320 may be modified to determine whether thepriority level falls within a range. For example, if the priority levelis associated with a character from “A” through “M”, then in block 320 adetermination may be made of whether the priority level of the datapacket falls within “A” through “M” or “N” through “Z”.

In block 330, the operation identifying the data packet associated withthe first designation (e.g., high priority) is placed in the structurewith the least number of data packets. If the structures are an equallength (e.g., have an equal number of data packets), then eitherstructure may be chosen. In block 340, the operation identifying thedata packet associated with the second designation (e.g., low priority)is placed in the structure with the most number of data packets.

When the operations are transfer operations, the invention ensures thatdata packets of similar priority are sent in the order that they wereissued to the device driver 118. The transfer operations for datapackets of similar priority are not necessarily stored on the samestructure. That is, one structure is not designated for transferoperations for high priority data packets, while another structure isdesignated for transfer operations for low priority data packets.

If multiple high priority data packets are requested, the structure thatis currently the shortest will change as transfer operations identifyingdata packets are added to the structure. In this case, the transferoperations for high priority data packets are added to the availablestructures in a round robin manner. This preserves the data packet orderand does not allow subsequent transfer operations to bypass any of thetransfer operations for high priority packets that are being added tothe structures.

Subsequent transfer operations for low priority traffic is added to thestructures behind these transfer operations for high priority requestsand added to the longer of the structures. If the structures are anequal length, then either structure is chosen, and the chosen structurebecomes the longer structure.

FIG. 4 illustrates an example set of transfer operations for high andlow priority data packets being placed into queues in accordance withcertain embodiments of the invention. The device driver 118 inserts aset of transfer operations 400 identifying data packets into queues 410,420 for access by the I/O controller 122. In the examples of FIGS. 4, 5,and 6, for ease of reference, it will be said that transfer operationsare stored in the queues, although data packets may be stored as well.For ease of-reference, a number following “Data Packet” indicates theorder of receipt of the transfer operation for the data packet of thespecified priority level. For example, Low Priority Data Packet 0, DataPacket 1, and Data Packet 2 are transfer operations that are received inorder of 0-1-2 and have low priority, and High Priority Data Packet 0and Data Packet 1 are transfer operations that are received in the orderof 0-1 and have high priority.

In this example, Low Priority Data Packet 0 identifies the firsttransfer operation received at the I/O controller 122 for transmission.Low Priority Data Packet 0 may be placed into either queue 410 or 420.For this example, Low Priority Data Packet 0 is placed into queue 1 410.Low Priority Data Packet 1 is received next and, since the data packetidentified by this transfer operation has a low priority, the transferoperation is placed into the queue with most transfer operations (i.e.,the longer queue), which is queue 1 410. Then, High Priority Data Packet0 is placed into the queue with the least transfer operations (i.e., theshorter queue), which is queue 2 420. High Priority Data Packet 0 islikely to be transferred before Low Priority Data Packet 1. That is, LowPriority Data Packet 0 will be removed from queue 1 410 and processed,then High Priority Data Packet 0 will be removed from queue 2 420 andprocessed, before Low Priority Data Packet 1 is removed and processed.This depends on how long it takes to process Low Priority Data Packet 0relative to when High Priority Data Packet 0 is added to queue 2 420.

Then, Low Priority Data Packet 2 is received and is placed into thelonger queue 1 410, and High Priority Data Packet 1 is placed into theshorter queue 2 420. For this example, the data packets identified bythe transfer operations are likely to be transferred over the network116 in the following order: Low Priority Data Packet 0, High PriorityData Packet 0, Low Priority Data Packet 1, High Priority Data Packet 1,Low Priority Data Packet 0.

Relative to the order of the transfer operations requested, the highpriority evens have bypassed lower priority transfer operations. Withlonger queue lengths, it would be possible for high priority operationsto by pass several low priority operations. When queue lengths are long,it is important for high priority data packets to receive preferentialtreatment in order to avoid high latency.

FIG. 5 illustrates an example set of transfer operations for mostly highpriority data packets being placed into queues in accordance withcertain embodiments of the invention. The device driver 118 inserts aset of transfer operations 500 identifying data packets into queues 510,520 for access by the I/O controller 122. In this example, High PriorityData Packet 0 is received at the I/O controller 122. High Priority DataPacket 0 may be placed into either queue 510, 520. In this example, HighPriority Data Packet 0 is placed in queue 1 510. High Priority DataPacket 1 is then placed into shorter queue 2, 520. High Priority DataPacket 2 is placed into the shorter queue 1 510. Low Priority DataPacket 0 is placed into the longer queue 1 510. High Priority DataPacket 3 is placed into the shorter queue 2 520.

In this example, the data packets identified by the transfer operationsare likely to be transferred over the network 116 in the followingorder: High Priority Data Packet 0, High Priority Data Packet 1, HighPriority Data Packet 2, High Priority Data Packet 3, and Low PriorityData Packet 0. This allows High Priority Data Packet 3 to bypass LowPriority Data Packet 0.

FIG. 6 illustrates an example set of transfer operations for mostly lowpriority data packets being placed into queues in accordance withcertain embodiments of the invention. The device driver 118 inserts aset of transfer operations 400 identifying data packets into queues 610,620 for access by the I/O controller 122. In this example, Low PriorityData Packet 0 is placed into queue 1 610. Then each other Low PriorityData Packet 1, 2, 3, 4, 5, 6 are placed into queue 1 610. High PriorityData Packet 0 is placed into the shorter queue 2 620. Depending on therate the transfer operations are requested and the operation processingtime, High Priority Data Packet 0, which was requested last for transfermay be the second data packet to be transferred.

In this example, the data packets identified by the transfer operationsare likely to be transferred over the network 116 in the followingorder: Low Priority Data Packet 0, High Priority Data Packet 0, LowPriority Data Packet 1, Low Priority Data Packet 2, Low Priority DataPacket 3, Low Priority Data Packet 4, Low Priority Data Packet 5, andLow Priority Data Packet 6. This allows a high priority data packet tobe given preferential treatment and to bypass multiple low priority datapackets. Thus, the high priority data packet was sent with lower latencythan the low priority data packets.

Thus, the described embodiments of the invention provide a method,system, and program for a technique for using multiple transferstructures (e.g., queues )of equal priority on an I/O controller toprovide preferential treatment of high priority data packets that are tobe transferred, for example, across a network.

Additional Embodiment Details

The described techniques for maintaining information on networkcomponents may be implemented as a method, apparatus or article ofmanufacture using standard programming and/or engineering techniques toproduce software, firmware, hardware, or any combination thereof. Theterm “article of manufacture” as used herein refers to code or logicimplemented in hardware logic (e.g., an integrated circuit chip,Programmable Gate Array (PGA), Application Specific Integrated Circuit(ASIC), etc.) or a computer readable medium, such as magnetic storagemedium (e.g., hard disk drives, floppy disks, tape, etc.), opticalstorage (CD-ROMs, optical disks, etc.), volatile and non-volatile memorydevices (e.g., EEPROMs, ROMs, PROMs, RAMs, DRAMs, SRAMs, flash,firmware, programmable logic, etc.). Code in the computer readablemedium is accessed and executed by a processor. The code in whichpreferred embodiments are implemented may further be accessible througha transmission media or from a file server over a network. In suchcases, the article of manufacture in which the code is implemented maycomprise a transmission media, such as a network transmission line,wireless transmission media, signals propagating through space, radiowaves, infrared signals, etc. Thus, the “article of manufacture” maycomprise the medium in which the code is embodied. Additionally, the“article of manufacture” may comprise a combination of hardware andsoftware components in which the code is embodied, processed, andexecuted. Of course, those skilled in the art will recognize that manymodifications may be made to this configuration without departing fromthe scope of the present invention, and that the article of manufacturemay comprise any information bearing medium known in the art.

In the described embodiments, certain operations were performed by thedevice driver 118. In alternative embodiments, these operations may beperformed by another device, such as the I/O controller 122 or byfirmware.

In the described embodiments, the data packets were transferred over anetwork 116. In alternative embodiments, the data packets may betransferred to local storage, to a peripheral device, or to anotherdevice without being transferred over the network 116.

In the described embodiments, two structures were described for storingdata packets. In alternative embodiments, more than two structures maybe maintained and data packets with high priority are added to theshortest structure, while data packets with low priority are added tothe longest structure. In yet other alternative embodiments, with two ormore structures available, data packets with a certain priority level(e.g., low priority) may be placed into a buffer and added to thestructures at a later time.

In the described embodiments, transfer operations were added tostructures. In alternative embodiments, any type of operation (e.g., astorage operation that is used to store data into a structure) may beadded.

The illustrated logic of FIG. 3 describes specific logic operationsoccurring in a particular order. In alternative embodiments, certain ofthe logic operations may be performed in a different order, modified orremoved. Moreover, steps may be added to the above described logic andstill conform to the described embodiments. Further, logic operationsdescribed herein may occur sequentially or certain logic operations maybe processed in parallel, or logic operations described as performed bya single process may be performed by distributed processes.

The foregoing description of the preferred embodiments of the inventionhas been presented for the purposes of illustration and description. Itis not intended to be exhaustive or to limit the invention to theprecise form disclosed. Many modifications and variations are possiblein light of the above teaching. It is intended that the scope of theinvention be limited not by this detailed description, but rather by theclaims appended hereto. The above specification, examples and dataprovide a complete description of the manufacture and use of thecomposition of the invention. Since many embodiments of the inventioncan be made without departing from the spirit and scope of theinvention, the invention resides in the claims hereinafter appended.

1. A method for adding an operation to a structure, comprising: if apriority level associated with a data packet identified by the operationhas a first designation that comprises a high priority, placing theoperation into a first structure storing a least number of operations;and if the priority level associated with the data packet identified bythe operation has a second designation that comprises a low priority,placing the operation into a second structure storing a most number ofoperations.
 2. The method of claim 1, further comprising: generating thepriority level associated with the data packet based on content of thedata packet.
 3. The method of claim 1, wherein the first structurecomprises a queue.
 4. The method of claim 1, wherein the secondstructure comprises a queue.
 5. The method of claim 1, wherein theoperation is placed into the first structure or the second structure bya device driver.
 6. The method of claim 1, further comprising: removingoperations from the first and second structures with a round robintechnique.
 7. The method of claim 1, wherein the priority levelassociated with the data packet has the first designation or the seconddesignation based on a range within which the priority level falls. 8.The method of claim 1, wherein the data packet comprises an Ethernetdata packet.
 9. A system for adding an operation to a structure,comprising: a processor; memory coupled to the processor; a firststructure storing a least number of operations; a second structurestoring a most number of operations; and at least one program executedby the processor in the memory to cause the processor to perform: (i) ifa priority level associated with a data packet identified by theoperation has a first designation that comprises a high priority,placing the operation into the first structure; and (ii) if the prioritylevel associated with the data packet identified by the operation has asecond designation that comprises a low priority, placing the operationinto the second structure.
 10. The system of claim 9, wherein the atleast one program further causes the processor to perform: generatingthe priority level associated with the data packet based on content ofthe data packet.
 11. The system of claim 9, wherein the at least oneprogram comprises a device driver program.
 12. The system of claim 9,wherein the at least one program further causes the processor toperform: removing operations from the first and second structures with around robin technique.
 13. The system of claim 9, wherein the prioritylevel associated with the data packet has the first designation or thesecond designation based on a range within which the priority levelfalls.
 14. A system, comprising: a first structure storing a leastnumber of operations; and a second structure storing a most number ofoperations; and a device driver to, (i) if a priority level associatedwith a data packet identified by the operation has a first designationthat comprises a high priority, place the operation into the firststructure; and (ii) if the priority level associated with the datapacket identified by the operation has a second designation thatcomprises a low priority, place the operation into the second structure.15. The system of claim 14, wherein the device driver is capable togenerate the priority level associated with the data packet based oncontent of the data packet.
 16. The system of claim 14, furthercomprising: an input/output controller to read the first structure andthe second structure.
 17. The system of claim 14, wherein theinput/output controller is capable to remove operations from the firstand second structures with a round robin technique.
 18. The system ofclaim 14, wherein the priority level associated with the data packet hasthe first designation or the second designation based on a range withinwhich the priority level falls.
 19. An article of manufacture comprisingone of hardware logic or a computer readable medium including a programexecutable by the hardware logic or a computer for adding an operationto a structure, wherein the program causes operations to be performed,the operations comprising: if a priority level associated with a datapacket identified by the operation has a first designation thatcomprises a high priority, placing the operation into a first structurestoring a least number of operations; and if the priority levelassociated with the data packet identified by the operation has a seconddesignation that comprises a low priority, placing the operation into asecond structure storing a most number of operations.
 20. The article ofmanufacture of claim 19, the operations further comprising: generatingthe priority level associated with the data packet based on content ofthe data packet.
 21. The article of manufacture of claim 19, wherein theprogram comprises a device driver program.
 22. The article ofmanufacture of claim 19, wherein the operations are removed from thefirst and second structures with a round robin technique.
 23. Thearticle of manufacture of claim 19, wherein the priority levelassociated with the data packet has the first designation or the seconddesignation based on a range within which the priority level falls. 24.An article of manufacture comprising a device driver program executableby a computer for adding an operation to a structure, wherein the devicedriver program causes operations to be performed, the operationscomprising: if a priority level associated with a data packet identifiedby the operation has a first designation that comprises a high priority,placing the operation into a first structure storing a least number ofoperations; and if the priority level associated with the data packetidentified by the operation has a second designation that comprises alow priority, placing the operation into a second structure storing amost number of operations.
 25. The article of manufacture of claim 24,the operations further comprising: generating the priority levelassociated with the data packet based on content of the data packet. 26.The article of manufacture of claim 24, wherein the operations areremoved from the first and second structures with a round robintechnique.
 27. The article of manufacture of claim 24, wherein thepriority level associated with the data packet has the first designationor the second designation based on a range within which the prioritylevel falls.